Amendments to the Specification: 

At the top of page 1 of the specification, insert the following paragraph: 

This is a continuation of applicatioii of Serial No. 10/425,775, filed on April 29, 2003, 
which is a continuation of application Serial No. 09/660,204, filed September 12, 2000, now U.S. 
Patent No. 6,558,166 Bl, which is a continuation of application Serial No. 09/141,804, filed on 
August 28, 1998, now U.S. Patent No. 6,168,440 Bl, which is a continuation of application 
Serial No. 09/003,979, filed on January 7, 1998, now abandoned, which is a continuation of 
application Serial No. 08/561,081, filed November 20, 1995, now U.S. Patent No. 5,735,694, 
which is a continuation of application Serial No. 08/290,014, filed August 12, 1994, now U.S. 
Patent No. 5,558,521, which is a division of application Serial No. 08/014,176, filed February 5, 
1993, now U.S. Patent 5,437,554. U.S. Patent 5,437,554, U.S. Patent No. 5,718,591, and U.S. 
Patent No. 6,193,521 are hereby incorporated by reference in their entirety. 

Please replace the paragraph beginning at page 3, line 15 with the following paragraph: 

The present multiple data item scoring method facilitates the speed at which data items 
are processed. The method efficiently organizes, groups, and displays the data items such that a 
resolver can simultaneously view multiple data items. In the method, a plurality of data items are 
received. The data items each comprise an electronic representation, such as a scanned image, of 
at least a portion of a person's work product. After the data items are received, the answers are 
organized into separate groupings, typically by evaluation expertise of the resolver. Finally, the 
method displays to a resolver a particular one of the groupings such that the resolver can 
selectively view and evaluate data items related to the resolver's resolution expertise. 

Please delete the paragraph beginning at page 3, line 21. 

Please delete the paragraph beginning at page 4, line 3. 

Please delete the paragraph beginning at page 4, line 15. 
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Please delete the paragraph beginning at page 5, line 12, and insert the following 
paragraphs: 

Fig. 12A is a flow chart of resolver monitoring and feedback. 

FIG. 12B is a flow chart of retrieval of information for resolver monitoring based on the 
type of performance feedback. 

Please delete the paragraph beginning at page 20, line 22, and insert the following 
paragraph: 

Figure 12A is a flow chart of typical resolver monitoring and feedback. The 
primary factors in monitoring performance typically include: (1) validity; (2) reliability; 
and (3) speed. Li monitoring these factors, the system promotes repeatability of scoring. 
These factors may be monitored by tracking a resolvefs performance against past 
performance of the resolver or against some known goal. 

On page 22, line 6, insert the following: 

FIG. 12B is a more detailed flow chart of step 121 from FIG. 12A. The flow chart 
of Fig. 12B shows the retrieval of information for resolver monitoring based on the type 
of performance feedback. At step 140, the system determines the type of performance 
feedback for monitoring the resolver, such as validity, reliability, or speed. At steps 141, 
the system retrieves information for providing performance feedback related to validity, 
which may include receiving first performance levels corresponding to a resolver's 
scoring of test answers at predetermined past intervals in time, and receiving a second 
predetermined performance level. At steps 142, the system retrieves information for 
providing performance feedback related to reUability, which may include receiving a first 
score corresponding to a first resolver's scoring of a test item during a first time period, 
and receiving a second score corresponding to the first resolver's scoring of the test item 
during a second time period. At steps 143, the system retrieves information for providing 
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performance feedback related to speed, which may include receiving a first scoring rate 
corresponding to a first resolver's scoring of test answers, including scoring test answers 
at predetermined past intervals in time; receiving an average scoring rate corresponding to 
a selected group of resolvers* scoring of the test answers; receiving a second scoring rate 
corresponding to a second resolver's scoring of the test answers; and receiving a second 
scoring rate corresponding to the first resolver's scoring of the test answers, including 
scoring of the test answers at a present time. 
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